WHAT IS CLAIMED IS: 



1 . A system for automatically decomposing an image sequence, 
comprising: 

5 providing an image sequence of at least one image frame of a scene; 

providing a preferred number of classes of objects to be identified within 
the image sequence; 

automatically decomposing the image sequence into the preferred number 
of classes of objects in near real-time. 

10 

2. The system of Claim 1 wherein providing the preferred number of 
objects comprises specifying the preferred number of classes of objects via a 
user interface. 

15 3. The system of Claim 1 wherein decomposing the image sequence 

into the preferred number of objects comprises automatically learning a 2- 
dimensional model of each object class. 

4. The system of Claim 3 wherein the model employs a latent image 
20 and a translation variable in learning each object class. 

5. The system of Claim 1 wherein automatically decomposing the 
image sequence into the preferred number of object classes comprises 
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performing an inferential probabilistic analysis of each image frame for identifying 
the preferred number of object class appearances within the image sequence. 

6. The system of Claim 5 wherein performing an inferential 
probabilistic analysis of each image frame comprises performing a variational 
generalized expectation-maximization analysis of each image frame of the image 
sequence, wherein the expectation-maximization analysis employs a Viterbi 
algorithm in a process of filling in values of hidden variables in a model 
describing the object class. 

7. The system of Claim 6 wherein the model describing the object 
class employs a latent image and a translation variable in filling in said hidden 
variables. 

8. The system of Claim 6 wherein an expectation step of the 
generalized expectation-maximization analysis maximizes a lower bound on a 
log-likelihood of an image frame by inferring variational approximations of 
variational parameters. 

9. The system of Claim 8 wherein a maximization step of the 
generalized expectation-maximization analysis automatically adjusts object 
model parameters in order to maximize the lower bound on the log-likelihood of 
the image frame. 
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1 0. The system of Claim 9 wherein the expectation step and the 
maximization step are performed once for each image in said image sequence. 



1 1 . The system of Claim 8 wherein automatic computation of the 
expectation step is accelerated by using an FFT-based inference analysis. 

12. The system of Claim 1 1 wherein the FFT-based inference analysis 
is performed on variables that are converted into a coordinate system wherein 
transforms applied to those variables are represented by shift operations. 

1 3. The system of Claim 8 wherein automatic computation of the 
expectation step is accelerated by using a Viterbi analysis. 

14. The system of Claim 1 wherein automatically decomposing the 
image sequence into the preferred number of object classes comprises 
performing a probabilistic variational expectation-maximization analysis. 

15. The system of Claim 14 wherein the variational expectation 
maximization analysis comprises: 

forming a probabilistic model having variational parameters 
representing posterior distributions; 

initializing said probabilistic model; 
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inputting an image frame from the image sequence; 

computing a posterior given observed data in said image sequence; 

and 

using the posterior of the observed data to update the probabilistic 
model parameters. 

16. The system of Claim 15 wherein the variational expectation- 
maximization analysis further comprises: 

outputting the model parameters. 

17. The system of Claim 15 further comprising incrementing to the next 
image frame in said image sequence and repeating the actions after initializing 
the probability model until the end of the image sequence has been reached. 

18. The system of Claim 1 further comprising a generative model which 
includes a set of model parameters that represent the entire image sequence. 

1 9. The system of Claim 1 further comprising a generative model which 
includes a set of model parameters that represent the images of the image 
sequence processed to that point. 

20. The system of Claim 19 wherein the model parameters include: 
a prior probability of at least one object class; and 
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means and variances of object appearance maps. 

2 1 . The system of Claim 20 wherein the model further comprises 
observation noise variances. 

22. The system of Claim 19 further comprising automatically 
reconstructing a representation of the image sequence from the generative 
model, wherein the representation comprises the preferred number of object 
classes. 

23. A computer-implemented process for automatically generating a 
representation of an object in at least one image sequence, comprising using a 
computer to: 

acquire at least one image sequence, each image sequence having at least 
one image frame; 

automatically decompose each image sequence into a generative model, 
with each generative model including a set of model parameters that represent at 
least one object class for each image sequence using an expectation- 
maximization analysis that employs a Viterbi analysis. 

24. The computer-implemented process of Claim 23 wherein the 
number of object classes learned for each image sequence is input via a user 
interface. 
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25. The computer-implemented process of Claim 23 wherein the model 
parameters of each generative model include an object class appearance map, 
and a prior probability of at least one object class. 

26. The computer-implemented process of Claim 25 wherein the object 
class appearance map for each generative model includes means and variances 
of that object class appearance map. 

27. The computer-implemented process of Claim 23 wherein a latent 
image and a translation variable are employed in determining said generative 
model. 

28. The computer-implemented process of Claim 23 wherein an 
expectation step of the generalized expectation-maximization analysis maximizes 
a lower bound on a log-likelihood of each image frame by inferring 
approximations of variational parameters. 

29. The computer-implemented process of Claim 23 wherein the 
maximization step of the generalized expectation-maximization analysis 
automatically adjusts model parameters in order to maximize a lower bound on a 
log-likelihood of each image frame. 



-47- 



30. The computer-implemented process of Claim 23 wherein the 
expectation-maximization analysis comprises: 

forming a probabilistic model having variational parameters 
representing posterior distributions; 

initializing said probabilistic model; 

inputting an image frame from the image sequence; 

computing a posterior given observed data in said image sequence; 

and 

using the posterior of the observed data to update the probabilistic 
model parameters. 

31 . The computer-implemented process of Claim 30 wherein the 
expectation-maximization analysis further comprises: 

outputting the model parameters. 

32. The computer-implemented process of Claim 23 wherein 
computation of the expectation step is accelerated using an FFT-based inference 
analysis. 

33. A computer-readable medium having computer executable 
instructions for automatically learning a generative model from an image 
sequence, comprising: 
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forming a probabilistic model having variational parameters representing 
posterior distributions; 

initializing said probabilistic model; 

inputting an image frame from an image sequence; 

computing a posterior given observed data in an image of said image 
sequence; 

using the posterior of the observed data to update the probabilistic model 
parameters; and 

outputting the model parameters. 

34. The computer-readable medium of Claim 33, further comprising 
repeating the actions after initializing said probabilistic model for all images of the 
image sequence. 

35. The computer-readable medium of Claim 33 further comprising 
using said output model parameters in metadata creation. 

36. The computer-readable medium of Claim 35 wherein a class mean 
is used to represent a class and used as metadata. 

37. The computer-readable medium of Claim 33 wherein the model 
parameters include: 

a prior probability of at least one object class; and 
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means and variances of object appearance maps. 

38. The computer-readable medium of Claim 33 further comprising 
using said output model parameters in one of the following: 
video partitioning; 
image sequence cut detection; 
motion characterization; 
scene representation; and 
definition of scene similarity measure 
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